Papers with memory savings
500xCompressor: Generalized Prompt Compression for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Prompt compression is important for large language models to increase inference speed, reduce computation cost, and improve user experience. |
| Approach: | They propose a method that compresses natural language contexts into a special token . they propose to reduce computations and memory costs by reducing the complexity . |
| Outcome: | The proposed method reduces computations and memory costs by 27-90% . it retains 70-74% and 77-84% of the LLM capabilities at high compression ratios . |